Bilevel Online Deep Learning in Non-stationary Environment

Recent years have witnessed enormous progress of online learning. However, a major challenge on the road to artificial agents is concept drift, that is, data probability distribution would change where instance arrives sequentially in stream fashion, which lead catastrophic forgetting and degrade performance model. In this paper, we proposed new Bilevel Online Deep Learning (BODL) framework, combine bilevel optimization strategy ensemble classifier. BODL algorithm, use an classifier, output different hidden layers deep neural network build multiple base classifiers, important weights classifiers are updated according exponential gradient descent method manner. Besides, apply similar constraint overcome convergence problem framework. Then effective drift detection mechanism utilizing error rate classifier designed monitor distribution. When detected, our algorithm can adaptively update model parameters via then circumvent large encourage positive transfer. Finally, extensive experiments ablation studies conducted various datasets competitive numerical results illustrate promising approach.